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ABSTRACT 

This multiple-site study assessed 48 prevention programs for 
high-risk youth funded by the Center for Substance Abuse Prevention, 
identifying program characteristics associated with strong substance abuse 
prevention outcomes. Data analysis indicated that substance abuse programs 
reduced rates of substance use, and the positive effects of program 
participation continued for at least 18 months after the program ended. Youth 
already using cigarettes, alcohol, and marijuana significantly reduced their 
use of substances after joining a prevention program. Gender played an 
important role in risk, protection, and substance abuse. Despite gender 
differences, programs that used multiple science-based practices identified 
in the study produced stronger and longer lasting effects for both boys and 
girls. Family, peers, school, community, and society protected against 
substance abuse. Connection to those five interwoven domains clearly played 
an important role in effective prevention efforts. Science-based program 
components produced consistent and lasting reductions in substance use. 
Communities with more opportunities for participation in prevention 
positively impacted substance use by youth. (Contains 11 references and 14 
figures.) (SM) 
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The Center for Substance Abuse Prevention (CSAP) in the Substance Abuse and Mental Health Services Administration 
(SAMHSA) is the Nations lead agency for substance abuse prevention. The Center funds community-based organisations 
universities, behavioral health providers, and public agencies to identify effective prevention programs and practices and dis- 
seminates findings, program models, and other prevention materials to practitioners and policymakers across the country. 

This document summari 2 es the findings of CSAP’s National Cross-Site Evaluation of High-Risk Youth Programs. This 
large multiple-site evaluation was designed to assess 48 prevention programs and to identify those program characteristics 
that are associated with strong substance abuse prevention outcomes. The rigor and relevance of the National Cross-Site 
Evaluation earned CSAP the Outstanding Study of the Year 2000 Award from the American Evaluation Association. 

The National High-Risk Youth Demonstration (funded from 1987 to 1995) has been one of the most ambitious and produc- 
tive of CSAP’s funding initiatives. In its early years, the demonstration focused on identifying promising approaches to pre- 
vention. As the demonstration matured, individual site and cross-site research produced knowledge about risk and protective 
factors related to substance use and helped to identify model programs for effective prevention. Research and experience in 
the High-Risk Youth Demonstration also contributed to the awareness of the importance of culturally sensitive, age-appro- 
priate and gender-specific programming. This progress in prevention theory and practice laid the foundation for the 
research reported in this document, the largest and most comprehensive of CSAP’s High-Risk Youth studies. 

In addition to individual-level information on substance use, risk, and protection, the evaluation includes detailed informa- 
tion on the nature and amount of prevention services in which each child participated, as well as systematic process informa- 
tion on the study programs. This monograph highlights the study’s contributions to the growing evidence that “Prevention 
Works. More specifically, this document highlights the study’s contributions to the understanding of how substance use 
develops in youth at high risk, and the risk and protective factors that contribute to or protect against substance use. The 
monograph also summari 2 es major findings concerning changes in substance use, risk, and protection as youth mature 
through adolescence, summari 2 es findings concerning the pathways between external and internal risk and protective factors 
and substance use during adolescence, and identifies implications for policies and programs designed to prevent substance 
use. 



This monograph is part of a series of ‘Toints of Prevention” publications that document the contribution of CSAP’s 
National Cross-Site Evaluation of High-Risk Youth Programs to prevention knowledge and provide science-based guidance 
for improved prevention policy and practice. 



Charles G. Curie, M.A., A.C.S.W 
Administrator 

Substance Abuse and Mental Health 
Services Administration 



Ruth Sanche 2 -Way, Ph.D. 

Director 

Center for Substance Abuse Prevention 
Substance Abuse and Mental Health 
Services Administration 
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Substance use is one of the Nation’s most pervasive, costly, and challenging health and social problems. The use, and partic- 
ularly the early use, of tobacco, alcohol, marijuana, and other illicit drugs is intricately entwined with serious personal and 
social problems including school failure, crime, family violence and abuse, and a host of additional problems that constitute a 
continuing national tragedy. For over a decade, the Center for Substance Abuse Prevention (CSAP) within the Substance 
Abuse and Mental Health Services Administration (SAMHSA) has been the Federal agency responsible for providing leader- 
ship in preventing the profound negative consequences of substance use. Important components of this responsibility 
include design and funding of demonstration substance use prevention programs, followed by evaluation of these programs 
to identify successful community prevention services and by dissemination of science-based guidance for strengthening pre- 
vention services in communities across the Nation. 

CSAP has made great progress in generating new knowledge about the design, implementation, and effectiveness of preven- 
tion strategies and activities. The most recent of these advances is CSAP’s completion of its National Cross-Site Evaluation 
of- High-Risk Youth Programs — a study that offers many lessons for prevention policy and practice. 

The High-Rish Youth Demonstpatlon Program 

CSAP’s High-Risk Youth (HRY) Demonstration Grant Program targeted substance use among at-risk youth. Since HRY 
began in 1987, CSAP has funded more than 400 High-Risk Youth projects that have created a wealth of knowledge about 
substance abuse prevention. Demonstration projects have been awarded to community-based organi 2 ations, umversities, 
behavioral health providers, and public agencies. These varied groups of grant recipients serve communities that are diverse 
in location, racial and ethnic composition, and level and nature of risk. 

The National Cross-Site Evaluation of High-Risk Youth Programs was a 5-year evaluation of more than 10,500 at-risk 
youth. Forty-eight HRY demonstration grant recipients across the Nation participated in the study. ^ These grantees were 
funded by CSAP to implement and assess programs to prevent and reduce the use of alcohol and other drugs among at-risk 
youth. 

Because the High-Risk Youth Demonstration Program took place over a decade, the grant process and the funded programs 
were able to mature and focus on promising practices and special prevention needs. Early evaluations of the HRY programs 
showed that the programs most successful in preventing substance use were those that clearly stated a theory about why the 
use began and continued; those successful programs often emphasized the concepts of risk elements and protective elements 
in the “web of influence” in the lives of at-risk youth (CSAP, 1993; CSAP, 1994). The experience gained over time also rein- 
forced research literature that identified special needs of specific populations, particularly girls and young women, who were 
relatively neglected in the early development of substance abuse prevention programming. As a result of this experience, 
and as the demonstration program matured, new grant recipients had to clearly outline the basic ideas of their interventions 
and were encouraged to emphasize risk and protection concepts (Sambrano, Springer, and Hermann, 1997). The 1994 and 
1995 High-Risk Youth Demonstration Programs included two special initiatives: the Female Adolescent Initiative ensured 
that projects addressed the special needs of girls and young women, and the Replication Initiative funded promising program 
models in diverse settings. 



6 

^ CSAP funded 94 programs in 1994 and 1995. Programs were not included in the Cross-Site Evaluation if they served children primarily under the age of 9. 

Approximately half of the 48 programs that participated in the Cross-Site study were funded in 1994 for 5 years, and the other half were funded in 1995 for 3 years. 
Q hese program were located throughout 22 states, including Alaska and Hawaii. 
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The National Cross-Site Evaluation of High-Risk Youth Programs created an unprecedented opportunity to learn more 

about effective substance abuse prevention for youth at risk. The evaluation was initiated with three major objectives: 

O Rigorously assess and explain how CSAP-funded HRY programs prevent and reduce substance use among high-risk 
youth. 

O Provide systematic information about the role of risk and protective factors in substance abuse prevention programs, 
including the linkages between risk and protection and substance use in the HRY target population, how important risk 
and protection are as objectives of prevention efforts, and how participating in the programs affected the experience of 
risk and protection for at-risk youth. 

O Systematically document the processes of prevention service delivery and of program implementation, and identify the 
program setting, design, and implementation characteristics that might explain patterns of program effectiveness. This 
objective was fundamental to developing science-based knowledge about which interventions and program characteris- 
tics contribute to achieving prevention objectives. 
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The National Cross-Site Evaluation of High-Risk Youth Programs provides strong evidence that substance abuse prevention 
programs in communities where youth face multiple risk factors do produce lasting reductions in substance use and have 
important positive effects on the lives of youth at risk. These results are not hypothetical. They represent real accomplish- 
ments by real programs operating in these communities. Major findings of the study include the following important out- 
comes: 

Substance abuse prevention programs produce statistically significant reductions in substance use and positively impact the 
lives of at-risk youth. 

1. Substance abuse prevention programs reduce rates of substance use. Youth who took part in CSAP fligh-Risk Youth 
prevention programs decreased their alcohol and marijuana use when compared to similar youth who did not participate 
in the programs. Substance use for participants in CSAP prevention programs was 12 percent less at exit than compari- 
son youth and 6 percent below comparison youth 18 months later. Positive effects of program participation continue 
for at least 18 months after the program ends. 

2. Youth already using cigarettes, alcohol, and marijuana significantly reduced their use of substances after joining a 
prevention program. As shown in figure 1, youth who had already started to use cigarettes, alcohol, and marijuana 
before entering a CSAP prevention program reduced their use after joining the program. Substance use by participants 
who reported prior drug use was 10 percent lower at exit than comparison youth, and use levels were 22 percent below 
comparison youth 18 months later. 

3. Gender plays an important role in risk, protection, and substance use. Boys experience lower levels of family supervi- 
sion than girls in the sample, and neighborhood conditions have more influence on their peer associations and substance 
use than for girls. Across all programs, substance use outcomes were more positive for boys than for girls at program 
end, but tended to fade by 18 months later. For girls, effects on substance use emerged later and lasted longer. Despite 
these differences, programs that used multiple science-based practices identified in the study produced stronger and 
longer lasting effects for both boys and girls. 

4. Family, peers, school, community, and society protect against substance abuse, fligh-risk youth who were connected to 
positive social environments such as school, family, peers, community, and society used substances less than those who 
lacked such connections. For youth at risk, connection to these five interwoven domains clearly plays an important role 
in effective prevention efforts. 

5. Science-based program components produce consistent and lasting reductions in substance use. Six program characteris- 
tics were scientifically verified in this study as significantly strengthening program impacts. When programs were charac- 
terked by five or more of these science-based practices, they consistently produced stronger and longer lasting positive 
effects than other programs. 

6. Communities with more opportunities for participation in prevention positively impact substance use by youth. 
Communities that gave young people opportunities to take part in prevention activities had greater positive impact on 
substance abuse among these youth than communities with fewer prevention opportunities. 
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CSAP’s National Cross-Site Evaluation of High-Risk Youth Programs found reductions in rates^ of alcohol and marijuana 
use by youth participating in a CSAP-funded prevention program relative to comparison youth who had limited opportunity 
to participate in prevention programs in their community. At exit, substance use^ for participants in CSAP prevention 
programs was 10 percent lower than for comparison youth. Eighteen months later (Figure 1),^ substance use was 6 percent 
lower for participants than comparison youth. These positive results were statistically significant and show that the effects 
of program participation continue to be felt for approximately 18 months after youth leave the programs. Even within the 
challenging conditions of this community-based evaluation design, the entire pool of participants across all programs 
decreased their use of alcohol and marijuana relative to similar youth who did not participate in the programs. 

Figure 1 

Trends in 30-Day Cigarette, Alcohol, Marijuana, and Substance Use 
For Participant and Comparison Youth 
(n = 5,195) 




Calculated for the 23 sites with low comparison group exposure to prevention programs. 



2 

Rates are measured in terms of the number of days in which a substance was used in the last 30 days; the numbers of days are combined across cigarettes, alcohol, and 
marijuana to calculate the substance use rate. 

Substance use combines use of cigarettes, alcohol, and marijuana to assess overall change in adolescent subst an ce use for the three most widely used substances in this 
population. 

Youth reported their frequency of substance use by indicating how many days they had used cigarettes, alcohol, or marijuana in the previous 30 days. Response cate- 
gories were (0), no use; (1), 1 to 2 days; (2) 3 to 5 days; (3), 6 to 9 days; (4), 10 to 19 days; and (5), 20 to 31 days. The numerical values of the categories were averaged 
for each group and adjusted for age, sex, e^c background, and baseline 30-day substance use. Analyses were conducted using Hierarchical Linear Modeling (HLM) 
(Bryk and Raudenbush, 2000). For these estimates of the magnitude of prevention effects the sample includes only those 23 sites in which comparison groups were 
shown to have limited opportunities for participation in prevention activities other than the study program (see Finding #6 below). 
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Alcohol Use. Alcohol use was the most widespread focal topic of substance use prevention in cross-site programs. 

Alcohol is more easily accessed and more prevalent than any drug in the study. Study programs produced statistically 
significant reductions in alcohol use rates for participants relative to comparison youth. Participant use rates were on 
average 11 percent lower than use rates of comparison youth over the 18 months following the end of the program. 

While the rate of reduction is small, it is statistically significant, which means that scientists view this as a real program 
achievement. Unlike cigarettes and marijuana, the reductions in alcohol use faded at the 18-month measurement point. 

Marijuana Use. Marijuana use begins at a later age than alcohol use, but rises extremely rapidly, particularly among males. 
Marijuana use is becoming a major concern in prevention as use proliferates, but its development and patterns of use are less 
well understood than for alcohol. The study programs produced lasting reductions in the rate of marijuana use by program 
participants relative to comparison youth. Program participants averaged a consistent 11 percent below comparison youth 
rates of marijuana use at program exit and 6 and 18 months after exit. 

Cigarette Use. Prevention of cigarette use is a well-developed field of knowledge. Although researchers note that ciga- 
rette prevention functions independendy of alcohol, marijuana, and illicit drug prevention, the cross-site programs produced 
statistically significant reductions in the rate of cigarette use by program participants. Program participants averaged 7 per- 
cent lower use rates than comparison youth over the three points in time when outcomes were measured. 

The majority of youth in this study are in the age range when substance use 
begins. Around three-fourths of all youth in this study (76%) reported no use 
of substances at baseline, making reduction of their use impossible. Accordingly, 
the fact that the overall reported use of alcohol and marijuana by participants 
was lower relative to comparison youth after participating in the program is an 
important result. 

A major objective of the Cross-Site Evaluation was to identify and explain how participation in prevention programs helps 
prevent substance use. This explanation compares results among youth in the program with results among similar youth 
from the same communities who were not participants. Although the pooled information on substance use across aU study 
programs shows that use rates for alcohol and marijuana were significantly lower for youth who participated in the programs, 
the size of this reduction is small. There are three factors that help explain why these pooled differences are small. 

First, using contrasts with comparison group youth to demonstrate program effectiveness presumes that the non-program 
youth do not have access to other prevention services. In communities that offer limited or no prevention programming, this 
presumption may be accurate. However, when the community has additional prevention programs, particularly those that 
may target youth similar to those in the cross-site programs, a true analysis of how effective the programs are must consider 
the fact that non-program youth may be exposed to as much or more prevention programming than program youth. This is 
why estimates of the overall magnitude of prevention effectiveness in this study are based on the 23 programs in which 
comparison group youth were shown to have only limited opportunity to participate in prevention programs other than the 
study program (see Finding #6). 
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Second, as discussed above, the large percentage of youth who did not use substances made it impossible to produce 
dramatic reductions in substance use among the full pooled study sample. Findings presented on the next page (Fin din g #2) 
demonstrate that the positive effects of prevention on substance use are much more evident for youth who already had 
initiated substance use when they entered the study. 



Third, the pooled effects of prevention across aU the programs are small because the study sample was composed of a 
broad range of programs implemented in real community conditions. The primary objective of the study was to generate 
information about what makes programs effective in these real settings. Not aU of the programs reduced substance use, 
though they often had other positive outcomes for the youth. Among those that did show positive effects on use, some 
were more effective than others. One of the most important outcomes of the study is the identification of program 
features that made the study programs more effective in reducing substance use among youth at high risk. These findings, 
summari 2 ed later in this overview (Finding #5), demonstrate that, while prevention currently benefits youth significantly, 
these benefits will be greatly strengthened as programs increasingly adopt the effective science-based practices generated by 
this study and other prevention research. 
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In high-risk communities, prevention programming must change behavior in youth who have already begun to use sub- 
stances, not just deter those who have not yet started. Most of the youth in the Cross-Site Evaluation reported no use of 
substances in the 30 days before the program began. Still, approximately 25 percent of youth reported some substance use 
at baseline, and use rates were higher for this high-risk population than for the general population of youth the same age. 
Participation in CSAP substance abuse prevention programs reduced use of cigarettes, alcohol, marijuana, and the combined 
use measure relative to comparison youth. Program participation produced significant reductions in substance use patterns 
and the positive effects continued over the course of the study. 



Figure 2 

Trends in 30-day Cigarette, Alcohol, Marijuana, and Substance Use Among Youth Who Initiated Substance Use Prior to Program Entry 

(n = 1,235) 




Note: ** Statistically significant at ,01 level; ^Statistically significant at ,05 level. 

Calculated for 23 sites with low comparison group exposure to prevention programs. 



Figure 2 presents the trends in use of cigarettes, alcohol, and marijuana and 30-day substance use for participant and com- 
parison youth who had already begun to use substances. Use of any of the three substances by participants who reported 
prior drug use was 10 percent lower at exit than comparison youth, and use levels were 22 percent below comparison youth 
18 months later. Furthermore, the positive impact of program participation on substance use was consistendy strong across 
the three most commonly used substances among, youth. Program participants’ rates of use of cigarettes in the last 30 days 
were an average of 17 percent below the rates for comparison youth at exit and at 6 and 18 months after program exit. For 
alcohol, participants’ rates were an average of 13 percent below rates for comparison youth across the three outcome time \ @ 
points; and participant rates of marijuana use in the last 30 days were an average of 18 percent below those of comparison 
youth at program exit and 6 and 18 months after exit. These findings indicate that the CSAP- funded prevention programs 
provided effective interventions for youth who have already been identified as initiating use. 
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Figure 2 also shows that, overall, participant youth reduced their use of all substances through the full period of the study. 
Given that widespread substance use typically starts and escalates among youth in the age groups included in this study, 
are called effective when they slow the rate of increased use among program participants. However, this analysis of youth 
who were already using at baseline demonstrates that program participation can actually reduce use rates. ^ 



Among the total sample, for both cigarettes and marijuana, participant youth report slightly higher levels of use at baseline 
than comparison youth. Furthermore, their use levels decrease more rapidly during the program period so that they are sub- 
stantially lower than those for comparison youth at program exit. For cigarettes and marijuana, the use gap between partici- 
pant and comparison youth continues to widen throughout the followup period. However, for alcohol use, the large positive 
impact on participants between program entry and exit fades substantially by the second followup. Alcohol is a widely avail- 
able and widely used substance among youth, and the implication is that a continued protective influence in the program 
environment may be necessary as a deterrent to alcohol use. 
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^ The sUght decrease in comparison group use in these analyses may be attributable to a statistical artifact called regression toward the mean. This is a tendency for a 
measurement (i.e., substance use) on which subjects are selected for an extreme value (i.e., substance use when the norm for the subject’s age is no use) to be more like 
the population average at a repeated measurement. Some of the youth who reported use in the 30 days prior to the baseline measurement are not habitual users, for 
example, and may not use in the 30 days prior to a second measurement. Controlling for this tendency is one reason for a comparison group, and the important point 
Q ere is that reductions in use scores at repeated measurement pints for treatment youth were larger than those for comparison youth. 
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Prevention practitioners and researchers have become increasingly aware of the differences between boys and girls in the way 
substance use develops and how well prevention works. The Female Adolescent Initiative was developed to fund 19 of the 
Cross-Site Evaluation programs partly because research suggested that current prevention programrning may be less effective 
for girls. The ability to compare boys’ and girls’ responses to substance abuse prevention participation across a large number 
of programs over time contributes new knowledge that will be helpful in developing programming for both genders. 

Prevention effects over all four points of data collection show a large gap (29%) in substance use rates between participant 
and comparison group boys at program exit (Figure 3). For girls, the difference between participant and comparison group 
use is only 3 percent at program completion. However, participant girls do significantly better than comparison girls 6 and 
18 months after program exit, with a 9 percent gap between participant and comparison group girls at the final measurement 
point. In contrast, use rates by participant boys move back closer to the rates of comparison boys after prevention programs 
end. In the study overall, substance use outcomes were more positive for boys than for girls at program’s end, but positive 
outcomes emerged later and lasted longer for girls. 



Figure 3 

Trends in 30-Day Substance Use by Gender 




Note: * Statistically significant at .05 level. 

Calculated for 23 sites with low comparison group exposure to prevention programs. 
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Despite these differences in the overall effects of program participation on boys and girls, programs that are characterked 
by multiple science-based program components (see Finding #5) identified in the study produced stronger and longer 
lasting effects for both boys and girls. Girls participating in the eight programs with five or more of these science-based 
components reported substance rates that were 24 percent lower than those reported by comparison girls at program exit, 
24 percent lower than comparison girls 6 months after program end, and 13 percent 18 months after their program ended. 
The pattern for boys is very similar. At the end of their programs, boys participating in strong programs reported substance 
use rates that were 28 percent lower than those reported by comparison boys, still 27 percent lower 6 months after program 
end, and 15 percent lower than those reported by comparison boys 18 months after program end. When programs are con- 
sistendy strong in design and delivery, they produce stronger and similar reductions in the rate of substance use for partici- 
pants relative to comparison youth for both girls and boys at high risk. In these weU-designed and implemented programs, 
the positive effects of substance use for high-risk boys are still significant 18 months after the program has ended. 
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The study extended knowledge about substance use, risk and protection, and prevention for boys and girls in additional ways, 
including the following: 

O There are some differences in risk and protection for boys and girls. Family supervision is a more consistent protective 
factor for girls than for boys and helps to reduce opportunities for girls to use. While schools are a central forum for 
the development of self-efficacy and protection against substance use among adolescents, the study suggests that schools 
may provide this protection more readily for high-risk girls than for high-risk boys. Peer association and substance use 
among boys are affected more by conditions of risk in the neighborhood. 

O For girls, programs that focus on behaviorally-oriented life skills topics are particularly important for sustaining positive 
effects on substance use throughout the 18-month followup period after programs end. This finding is particularly 
important for designing programs for girls because the female adolescent programs in this study tended to focus on 
affective programming concerning issues of identity and self-concept more than the life skills topics which were demon- 
strated to be more effective. For boys, participation in programs that emphasize methods of delivery that involve youth 
interactively with peers are particularly important for strengthening program effects on substance use. 
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Over the past decade, risk and protective factors have become a dominant part of the theory behind substance use preven- 
tion and associated problems among youth. In its most recent funding years, CSAP used the risk-and-protection framework 
to help document the need for service and to develop program objectives among grant recipients. Risk and protective fac- 
tors form a “web of influence” for each child, an interwoven backdrop that influences each child’s life. A growing body of 
research has documented this ‘Sveb of influence” through which community, family, school, and peer factors may put youth 
at risk for substance use (CSAP, 1999b). The web consists of the forces at work in five domains: family, peers, school, 
community, and society. The evaluation confirms that family, peers, school, and community are all important influences 
on substance abuse among high-risk youth. Efforts to strengthen families (by encouraging communication, appropriate 
supervision, and positive norm setting) and to strengthen schools as caring communities (to improve school bonding and 
opportunities for meaningful achievement) increase the protection of youth against substance use. 

The National Cross-Site Evaluation was designed to develop information that is useful to prevention programs as they are 
developed and implemented. For this purpose, it is important to distinguish between risk and protection factors that are 
“external” to youth (in their environment) or “internal” to youth (the way that they process, interpret, and respond to their 
environment). Program activities designed to reduce risk or promote protection in external environments like community, 
school, and family differ from prevention strategies that work directly with youth to develop internal protective factors. 
Analysis of programs in this study indicates a focus on internal risk and protective factors that show up in the attitudes, 
beliefs, and orientations of youth. Program activities and objectives emphasize strengthening individual characteristics in 
young people themselves. One of the important contributions of the National Cross-Site Evaluation is to clarify the rela- 
tionship between external and internal risk and protection, prevention interventions, and substance use. 

Types of Risk and Protection 



External risk and protective factors can be defined as the characteristics of young people’s circumstances that give them the 
opportumty or influence them to use substances or to avoid them. The study measured four external factors: 

O Family supervision 

O School promotion of prevention activities 

O Participation in organized community prevention activities 

O Social disorganization and risk in young people’s neighborhoods 
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Adolescents move from protection toward risk as they get older (Figure 4), with a decided decline in two of these factors: 
family supervision and school prevention. This pattern is not surprising because youth mature and family supervision dimin- 
ishes. Involvement in community activities declines least, suggesting an ongoing opportunity for community-based preven- 
tion. As youth become more independent, they form associations in the community. The challenge for prevention is to help 
make these associations positive. 



Figure 4 

External Risk and Protective Factors by Age at Program Entry 
(N = 10,473) 




Internal risk and protective factors refer to young people’s emotional and developmental processes. Internal risk and protec- 
tive factors shift in terms of their relative importance during the age span included in the National Cross-Site Evaluation 
population. The study measured six internal risk and protective factors: 



o 


School bonding^ 


o 


Family bonding 


o 


Belief in self 


o 


Self-control 


o 


Self-efficacy 


o 


Social confidence 





Around age 12, most high-risk youth begin to move from the protective toward the risk ends of these measures, bottoming 
out at age 15 (Figure 5). The exception is social confidence, which increases slighdy from preadolescence to the mid-teen 
years. The drop toward greater risk in the remaining orientations during the middle and early high school years indicates that 
internal protective orientations are also lessening during these years. The change is most pronounced with school and family 
bonding. The important implications in these patterns are that interventions need to be age-specific and offered at the time 
during the maturing process when they will have the greatest impact, and that meaningful associations with family and school 
should be prominent foci of adolescent interventions. 
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Q or some analyses, school bonding and self-efficacy are combined into a single 




measure of school connectedness. 
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Figure 5 

Internal Risk and Protective Factors by Age at Program Entry 
(N = 10,473) 




Connectedness and Substance Use 

The cross-site programs targeted risk and protective factors because researchers expect these factors will reduce substance 
use and other negative behaviors. The study findings on the association of individual protective factors and substance use 
indicate that ties to family and school are central to prevention of substance use (Figure 5). The analysis suggests that 
individual protective factors that begin and help maintain connectedness with positive external environments (family bonding 
and school bonding) are more strongly associated with lower levels of substance use than other internal factors measured in 
the study (belief in self, self-efficacy, social confidence, and self-control). These data, and the more complex findings of the 
risk and protective factor modeling in the smdy, are consistent with a recent research review concluding that “closeness to 
and connectedness with family and school were the most saUent, cross-cutting protective factors” (Resnick, 1999). 



Figure 6 

Summary of Associations Between Internal Risk and Protective Factors and Substance Use by Gender* 

(N = 10,339) 




Bar length indicates the strength of the negative association between the factor and substance use — in other words, 
thedegree to which the factor protects against substance use. Longer bars indicate stronger protective relationships. 



The data also indicate important differences between boys and girls in linking protective factors and substance use. Across 
all protective factors, the relationship between internal risk and protective factors and substance use is stronger for girls than 
for boys. This difference is especially true for family bonding and self-efficacy. 
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What Are the hllajor Pathways to Substance Use? 

To advance understanding of the relationship between risk and protective factors and substance use, the CSAP study team 
used a statistical technique to estimate the pathways that may link risk and protective factors to substance use (Figure 7). 
Although the technique does not test cause-and-effect relationships, it does indicate relationship patterns that are consistent 
with protective influences. 
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Two new variables were constructed for this analysis: 

O School connectedness combines the school bonding and self-efficacy factors to form a single measure representing the 
importance of the school as a setting for realizing self-efficacy in youth. It suggests that real connectedness to school is 
associated with a recognition of the ability to achieve. 

O Family connectedness combines the original family bonding measure with a family communication measure to form a 
single measure representing the importance of family as a forum for safe and supportive interactions during the develop- 
ing years. 

The resulting model (Figure 7) shows that family and peers are important influences on substance use and that school and 
community conditions also influence use. The width of the arrow reflects the strength of the association. Numbers along 
lines indicate the strength of the relationship. 



Figure 7 

Pathways of Influence Among Risk and Protective Factors and Substance Use 
(N = 10,473) 
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The model provides a plausible understanding of the way risk and protective factors from different domains in the ‘Veb 
of influence (CSAP, 1999) interact in their association with substance use. These relationship structures, which are highly 
consistent with the data produced by the National Cross-Site Evaluation, support the following interpretation: 



O Family connectedness is key in preventing substance use. Where family connectedness is high, family supervision and 
parental attitudes have a strong influence on which peers young people choose to associate with and also influence the 
decision to use substances or not. VCTiere family connectedness is high, parental influence can be significant, even when 
it conflicts with peer attitudes and behaviors. These findings substantiate the critical role parents play in shaping peer 
associations of their children and in direedy influencing behavior. 

O School is a critical forum for developing healthy adolescent behaviors. When school connectedness and school success 
are linked, they are strongly associated with youth choosing peers who do not use substances and choosing not to use 
substances themselves. 

O Peers have a direct relationship to individual substance use. Youth whose best friends use substances are very likely 
to use substances themselves. In addition, much of the influence of family, school, and community on individual 
substance use works through the mediating influence of the selection of peers. 

O In the commumty domain, neighborhood risk and peer substance use together form an influence on personal use. 
Involvement in protective community activities is associated with high performance in school. Community risk and 
protection shapes the opportunities that are mediated by internal factors and peer associations. 

Family and school represent powerful protective factors when they offer youth meaningful involvement: involvement that is 
challenging, provides recognition, and is rewarding. 

In summary, this model begins to tease out the dynamics of the 
‘Veb of influence” to explain the relationships between external 
risk and protection, internal risk and protection, and substance use. 

Although distinct concepts, external and internal protection are 
tighdy interwoven in the dynamics of behavioral development in 
youth. Connectedness lies at the core of the model — specifically, 
the connectedness to family represented in family bonding and the 
connectedness to school represented in school connectedness. 

These two external environments represent powerful protective 
factors when they offer youth meaningful involvement — involvement 
that is challenging, provides recognition, and is rewarding. 

The model also indicates the importance of comprehensive and persuasive prevention activities and resources in the 
community. Although the Cross-Site Evaluation programs focus on the internal protective factors, clearly their efforts 
will be more effective if famihes are strengthened (to provide communication, appropriate supervision, and positive 
norm setting). Similarly, school bonding will be more meaningful if schools are strengthened as caring communities 
(Battistich et al., 1996). 
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The evaluation included 48 programs implemented in communities at high risk. These programs experienced all the 
challenges of implementing programs in these settings, and they approached these challenges using differing prevention 
strategies and program management approaches. This diversity was important to the research objectives of the study 
because it provided an opportumty to learn from the differences among programs. Not all of the programs succeeded, 
and of those that did achieve their objectives, some were more successful than others. 

The evaluation identified and verified six program components that produced statistically significant improvements in the 
degree to which programs achieved reductions in the rate of substance use by participants relative to comparison youth in 
each site. Figure 8 identifies the six program characteristics. The height of the bar indicates the relative magnitude of the 
effectiveness (effect size) for programs with that characteristic; the higher the bar, the stronger the effectiveness of the 
programs with that characteristic. The identification of these characteristics of effective prevention programs is an important 
contribution to science-based understanding of how to make prevention programs more effective for high-risk youth. 

Figure 8 

Summary of Program Characteristics Producing Statistically Significant Improvements in the Effectiveness of Programs in Reducing Rates of 30-Day Substance Use 



D.18 




Ufe-Skills Focused Coherent Intense Contact 

The figure demonstrates that the following program characteristics are important contributors to effective reduction of the 
rates of substance use among youth at high risk. These findings are explained and elaborated in Points of Prevention 
Monograph No. 3, Findings on Designing and Implementing Effective Prevention Programs for Youth at High Risk. 

O Life Skills Focus. Programs in the study that emphasized the promotion of attitudinal and behavioral life skills often 
identified as protective factors were more effective in reducing substance use than programs that emphasized knowledge- 
only or affective objectives such as self-esteem. This finding confirms that the life skills content found to be effective in 
previous studies (Tobler, 1986; Tobler and Stratton, 1987; Tobler et al., 2000) is also effective in reducing substance use 
for high-risk youth. 
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O Emphasis on Building Connectedness. Prevention programs in the study that emphasized the use of program 
delivery methods designed to strengthen connectedness to positive peers and adults through team and interpersonal 
activities were more effective than programs that emphasized other delivery methods. This finding provides a deeper 
understanding of the greater effectiveness of interactive versus classroom style delivery methods than has been found 
in earlier prevention research (Tobler, 1986; Tobler and Stratton, 1987; Tobler et al., 2000). 

O Coherent Program Design and Implementation. Prevention programs that selected strategies, implemented 
activities, and trained staff within a clearly articulated and coherent prevention theory were more effective than 
those that were designed with less clarity and consistency. This finding underlines the importance of effective 
program design and management in making prevention programs effective. 

O Introspective Orientation. Prevention programs that emphasized introspective learning approaches were more 
effective than programs that did not utilize this perspective. These approaches encourage youth to examine their 
own attitudes and behaviors and how they impact themselves or others in social contexts that are relevant to them. 
This finding adds to understanding of how to effectively structure interactive program delivery methods. 

O Intensive Contact. Programs with more intense contact (i.e., approximately 4 or more hours per week) achieved 
more positive outcomes than those with less intense contact. This program feature was more important for program 
effectiveness than either the length of the program or the total number of contact hours. 

O After-School Setting. CSAP-funded prevention programs that offered after-school hours — ^when youth are most 
at risk for substance use — ^were more effective in reducing substance use for high-risk youth than those deUvered 
exclusively within school hours. The greater effectiveness of after-school programs can be largely explained by their 
ability to incorporate program features that the study identified as contributing to greater positive effects on substance 
use across all study sites. The constraints of classroom delivery meant that in-school programs were less able to 
incorporate these features. 

The positive effects of these science-based practices are demonstrated in the substance use rates of youth participating in 
programs that incorporate all or most of these practices. Figure 9 shows the relative trends for youth who participated in 
the seven study programs that were characterized by at least five of the practices described above."^ The programs provide 
strong prevention benefits that last throughout the 18-month followup period of the study. 



Through identifying the design and implementation features that characterize more effective prevention in real commumty 
settings, the evaluation has contributed important science-based guidance for strengthening future prevention initiatives 
and programs. 

Figure 9 

Trends in 30-Day Substance Use: Seven Representative Sites with Positive Program 
Characteristics (n = 1759) 




^ An eighth site with 5 positive characteristics was not included in this analysis because it was an outlier wjt^ f^pect to the magnitude of its positive effects. Including that 
site would have created a graph unrepresentative of typical program effectiveness. 4r 
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Since the National Cross>Site Evaluation was studying youth in actual community settings, evaluators considered 
that some of the youth in comparison groups would have the opportunity to enter prevention programs other 
than the CSAP program under study. To identify the degree to which this occurred, and the effects on study find- 
ings, data were gathered on the availability of prevention prograrnrning in the school and community environ- 
ments of study youth. When study communities were divided into those that offered higher and lower opportuni- 
ty for participation in prevention programs, an important finding emerged. 

Results from the National Cross-Site Evaluation of High-Risk Youth indicated that comparison youth in sites with high 
opportunity for prevention participation have lower rates of use than youth in sites with fewer available prevention 
resources. This supports the cross -site evaluation conclusion that ^Trevention Works” for high-risk youth. The results indi- 
cate that prevention effects in sites with higher availability of prevention services will be underestimated because comparison 
youth in these field settings also benefit from prevention. To make sure the findings of the study are accurate, estimates of 
the magnitude of prevention effects on youth are made using the 23 sites in which comparison youth had lower opportunity 
to participate in prevention programs. This finding also suggests that prevention effectiveness is often underestimated in 
evaluation studies where comparison youth have opportunities to participate in prevention activities within the community. 

Information on in-school and community prevention programming indicates that 
many of the HRY programs operate in communities that offer comparison youth 
opportunities to participate in several different types of prevention programming. 

When the data were pooled for all youth across the 48 sites, the data showed that 
there were nearly as many protective factors in place for comparison youth as for 
HRY program participants. Those protective factors include school prevention 
programming and involvement in organi 2 cd community activities such as youth 
organi 2 ations, after-school classes, and organized athletics. In 20 programs, partic- 
ipation in prevention and organized activities was actually higher for comparison 
youth than for HRY program youth, because programs other than the CSAP-fimd- 
ed program were available in their communities. 

To address the degree of comparison group exposure to prevention services, the evaluation researchers divided study sites 
into two categories. The 23 sites in which comparison group youth had litde access to prevention services were termed low- 
exposure sites. The remaining 25 sites at which comparison group youth had plentiful opportunities for exposure to organ- 
ized prevention services were termed high-exposure sites. The expectations were, first, that this study might identify the 
degree of influence of prevention programming on comparison youth in the 25 high-exposure sites. The second expecta- 
tion was that it would be harder to detect the impact of program participation on participants in high-exposure sites than in 
low-exposure sites. 
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Looking at comparison youth only, youth in high-exposure sites reported less substance use from baseline to exit than did 
youth in low-exposure sites (Figure 10). More specifically, comparison group youth who had greater opportunities to take 
part in prevention activities reported 57 percent to 68 percent smaller increases in alcohol, marijuana, and cigarette use than 
the comparison youth who had fewer opportunities for prevention participation. This finding among comparison group 
youth indicates that widely available prevention activity reduces the degree to which youth begin and increase substance use 
in the critical adolescent years. 



Figure 10 

Average Change in Substance Use From Program Entry to Exit for Comparison Group Youth: 
Sites With Low And High Comparison Group Exposure to Prevention Services 
(n = 4,341) 
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These results have major implications for the interpretation of findings in the Cross-Site Evaluation: 

First, the results indicate that comparison youth in sites with high exposure to prevention services benefit from community 
exposure to those services. This pattern strengthens support for the National Cross-Site Evaluation conclusion that 
‘Trevention Works.” Where comparison group youth are exposed to high availability of community prevention, they 
report smaller increases in substance use as they get older. 

Second, the results suggest that the study underestimates the effects of the study programs in sites where exposure to pre- 
vention activities is high. In these sites, comparison youth also benefit from community prevention services. To make sure 
this analysis is as objective as possible, only sites with low comparison group exposure to prevention services are used when 
the purpose of analysis is to estimate the effectiveness of CSAP-funded programs. Where the focus of the analysis is the 
relationship of program characteristics to program effectiveness, the full sample of sites is used because the additional infor- 
mation provided by high-exposure sites is important. It is important to recognize, however, that analyses using the full site 
sample underestimate the effectiveness of programs in high-exposure sites. 

For the larger understanding of prevention, this finding reinforces the positive effects of prevention programming in the 
community. Furthermore, it suggests that prevention’s effectiveness has probably been underestimated in research where 
community prevention programming was available but not measured or considered in interpreting study results. 
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The CSAP National Cross-Site Evaluation of High-Risk Youth Programs provides science-based guidance for improved 
prevention policy and practice. For policymakers, this rigorous study of a broad range of programs implemented in actual 
communities demonstrates the effectiveness of prevention for at-risk youth. 

The data re-emphasize that substance abuse prevention programs are a proactive step toward reducing substance use or 
keeping it from beginning. Among the findings from the HRY evaluation that will help policymakers are: 

O Evaluations like the National Cross-Site Evaluation enable policymakers to see who benefits from prevention 
programming and whether some strategies or approaches will be more effective than others. 

O Policymakers benefit when they understand how to incorporate risk and protection objectives into policy design and 
funding decisions. Prevention professionals benefit from this knowledge in designing projects, programs, and activities. 

O The Cross-Site Evaluation identified and tested the contribution of science-based prevention practices so that policy- 
makers can facilitate the design and implementation of the most effective programs. 

O Policies and programs that increase prevention opportumties in communities, strengthen families, and strengthen schools 
as environments to which youth are meaningfully connected (Battistich et al., 1996) are important complements to 
prevention programs that work with youth direcdy. 

O Substance abuse prevention programs designed for specific populations get results and are an effective part of Federal 
drug control policy. 

O CSAPs investment in development of sound prevention programs benefits youth; prevention works. 

For prevention practitioners, the study provides lessons on how prevention programming can be refined and strengthened 
in several areas. The findings have important implications for prevention design and implementation. By contributing to 
understanding the factors that prevent substance abuse in youth, and producing science- based lessons about effective 
prevention strategies and program practices, this major research effort holds the promise of more effective prevention 
and more positive futures for America’s youth who are high risk. 



For teachers, counselors, and other substance abuse prevention practitioners, this research can help identify “blueprints” of 
effective practice to help guide their efforts to promote positive youth development. Prevention professionals can use the 
information generated by CSAP’s study to improve their aim at this moving target of approaches. This diverse high-risk 
study sample provides an excellent proving ground for identifying connective paths among risk and protection factors and 
substance use. It also provides a basis for assessing the appropriateness of specific prevention objectives and practices in a 
variety of settings. Among the findings from the High-Risk Youth Evaluation that will help practitioners nationwide are: 



O Prevention is most effective when it focuses on reducing risk and/or strengthening protection in young Uves. 

O Programs that focus on developing Ufe skills are more effective in reducing substance use than programs that emphasize 
other program content. 



O Programs that involve participants interactively are more effective in reducing substance abuse than programs that rely 
on passive classroom-style teaching. 



O 

O 

O 



Programs that are designed and implemented with a clear and coherent prevention approach are more likely to have pos- 
itive impacts on participating youth. 

Young men’s and young women’s risk and protection influences differ, pointing to the need for differing gender-based 
strategies. 

Programs that combine life skills, interactive delivery, intensive participation, and strong implementation consistendy 
produce stronger and longer lasting positive effects on substance use. 
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The Cross-Site Evaluation implemented a comprehensive and rigorous study design marked by six major features: sample 
diversity, comparison samples, a common survey instrument, multiple measurement points, program contact data, and 
program-level measures. Each is explained below. 



O 



Diverse Program Sample. Included in the sample of programs are 48 sites funded through three initiatives of the 
HRY demonstration during 1994 and 1995. Programs serving children primarily under the age of 9 were excluded. 
Within these limits, programs were selected to ensure coverage of different regions of the country (Figure 11), differing 
funding initiatives, and differing target population characteristics (e.g, age, gender, and racial and ethnic identification). 
The quality of the program design or implementation was not a selection criterion. The objective was to study pro- 
grams that represented the range of strategies, capabilities, and participation in the various funding initiatives, not to 
study only promising or proven programs. 



Figure 11 

Location of Programs 
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O Participant and Comparison Youth Samples Within Each Site. Across the 48 sites, the study involved more than 
10,500 youth: 6,031 HRY program participants and 4,579 similar youth from the same communities who did not receive 
services from the participating CSAP programs. Respondents were between 9 and 18 years of age and of diverse racial 
and ethnic backgrounds. Approximately two- thirds of the respondents were girls, reflecting the inclusion of 19 female 
adolescent programs in the sample. This design allows the effectiveness of programs to be tested by comparing changes 
in risk, protection, and substance use oyer time in program participants with changes in youth who did not receive 
program services. 
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O Common Survey Instrument. All responses from study participants were collected using CSAP’s National Youth 
Survey, a self-report instrument developed for this study. The instrument includes sections titled “Facts About You” 
(personal attributes), “How Do You Feel?” (internal risk and protection), and “Cigarettes, vUcohol, and Other Drugs” 
(external risk and protection; problem behavior). The questionnaire was available in both English and Spanish. 

O Four hleasurement Points. Study participants were asked to respond to the survey at four different times during the 
research study. The responses taken at the program entry (also known as “baseline” data) and program exit (exit data) 
were designed to collect information on short-term program effects. Such data are comparable to much of the existing 
information on prevention programs. In addition, the National Cross-Site Evaluation design included two measurement 
points beyond program exit — one approximately 6 months later and a final measurement about 18 months after youth 
exited the program. 
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O Detailed Program Contact (Dosage) Data. Detailed information on both program content and program contact was 
collected for each program participant. Categories of program content included substance use information, educational 
support, personal and social skills development, positive recreation and enrichment, and social and emotional support. 
Categories of content-delivery method included classroom presentation, experiential, supported group, and one-on-one 
interaction. Information was gathered on more than 217,000 individual program-related contacts. 

O Program-Level Measures. Measures of program setting, planned prevention strategies, and the capacity of the 
administering organization to implement the program (e.g, staffing, adequacy of training and guidance, provision of 
adequate resources) were developed for each site. A unique site visit measurement tool was developed that combined 
both quantitative and qualitative information. Common coding schemes were developed to allow cross-site comparison. 
These program-level measures will enable the analyses of those characteristics of program setting, prevention strategy, 
program dosage, and program implementation where programs have had positive outcomes. 



The Cross-Site Evaluation design has generated a large and comprehensive data set that supports analyses of prevention 
effectiveness from a variety of perspectives, including analyses that distinguish between program effects that occur while 
youth are in the program and those that occur after the program ends. The multiple levels of data allow prevention 
effectiveness to be assessed on both a program and individual basis. Program-level fin ding s permit assessments and 
comparisons of program effectiveness. Individual-level findings support conclusions about the effects of prevention 
activities on youth across programs. 
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As part of the grant application process, CSAP grant recipients describe how they 
recruit youth who are at high risk for substance use. Most typically, recruitment focuses 
on communities that are characterized by high levels of risk, such as poverty, crime 
rates, or ambient substance use rates, that are known to be associated with higher than 
average levels of substance use in youth. Figure 12 compares use rates of 12- to 17- 
year-olds in the smdy sample with those of youth who participated in the 1998 National 
Household Survey on Drug Abuse (NHSDA), a randomly sampled general population 
survey of persons age 12 years and older. 




The youth in the cross-site sample reported higher use for all substances within all age groups than did the general popula- 
tion. Also, the use rates for cross-site youth rise more rapidly with age than do those in the general population (Figure 12), a 
pattern that is particularly noticeable for marijuana. Although the circumstances of the NHSDA and the National Cross-Site 
Evaluation data collection are not identical, this comparison suggests that the cross-site programs served youth who were at 
higher risk for initiating substance use when young. 



Fig;ure 12 

Comparison of NHSDA and HRY Substance Use Rates; Percentage of Youth Using 
Cigarettes, Alcohol, and Marijuana in the Past 30 Days 
NHSDA sample size: 6,778 
HRY sample size: 7,245 
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The majority were not substance users at the baseline measurement, which was entry into the prevention program. Figure 13 
displays, by age and sex, any self-reported use of cigarettes, alcohol, or marijuana in the 30 days before baseline data collec- 
tion of youth in the study. Substance use is very low until about age 12 and then rises rapidly through the early teen years. 
Rates are somewhat lower for girls, particularly after age 14. 



Fig;ure 13 

Percentage of Youth Reporting Substance Use in Past 30 Days at Program Entry by Age 

(N = 10,473) 
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The majority of youth in the sample (76%) reported no substance use at baseline. This 
finding indicates that youth are being recruited at an appropriate age for prevention, 
before they have initiated use, but it also presents an analysis problem because youth 
who do not use cannot ‘‘do better’’ at later points in time. Such a high occurrence of 
non-use at the baseline measurement creates a “floor effect” that makes detecting 
positive program effects more difficult. As noted earlier, this effect is one reason for 
the small magnitude of measured prevention effects in this study and other studies of 
programs serving young adolescents. In this study, separate analyses of youth who had 
already initiated use found larger positive program effects. 

Because 19 of the sample programs were funded through the Female Adolescents 
Initiative, two-thirds of the sample are girls. The cross-site programs targeted a diversity of racial and ethnic groups (Figure 
14). More than one-third of the youth are African American; approximately one-fourth are Hispanic; and the remaining 
youth are relatively evenly distributed among Native American (13%), non-Hispanic White (12%), and Asian or Pacific 
Islander (11%). 

Figure 14 

Distribution of Study Participants by Race and Gender 

(N = 10,473) 





In summary, the young people represented in the National Cross-Site Evaluation represent a diverse sample of youth at risk, 
with a large percentage of females and a large sample of youth who had not yet initiated substance use. The diversity of 
this sample provides an opportumty to test findings on substance use, risk and protective factors, and prevention program 
effectiveness across different sub-groups of youth at risk. 
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CSAP’s National Cross-Site Evaluation of High-Risk Youth Programs has contributed important new information that 
documents the effectiveness of prevention programming, and provides specific guidance on how to strengthen prevention 
programs in communities across the Nation. This monograph has identified and elaborated on six major findings that 
have implications for prevention policy and practice. Not only does the study demonstrate that “prevention works,” but 
goes further to identify what works and why. For prevention practitioners, the study provides lessons for refining and 
strengthening prevention programming in several areas. The evaluation also has produced important implications for 
prevention design and implementation. 

An initial objective of the National Cross-Site Evaluation was to use rigorous research to examine the experiences of 
prevention programs to extract lessons for improving future programming. This monograph summarizes findings that are 
important for realizing these objectives. Further analyses in a “Points of Prevention” monograph series describe in more 
detail the findings of the study and their many implications for prevention practice. The final step belongs to the prevention 
policymakers and practitioners whose charge it is to put these lessons to work for the benefit of our Nation’s youth. 
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